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We present a general formalism to make the Replica-Symmetric and Replica-Symmetry-Breaking 
ansatz in the context of Kikuchi's Cluster Variational Method (CVM). Using replicas and the 
message-passing formulation of CVM we obtain a variational expression of the replicated free energy 
, of a system with quenched disorder, both averaged and on a single sample, and make the hierar- 

■ chical ansatz using functionals of functions of fields to represent the messages. We obtain a set of 

integral equations for the message functionals. The main difference with the Bethe case is that the 
functionals appear in the equations in implicit form and are not positive definite, thus standard 
iterative population dynamic algorithms cannot be used to determine them. In the simplest cases 
the solution could be obtained iteratively using Fourier transforms. 

We begin to study the method considering the plaquette approximation to the averaged free energy 
of the Edwards- Anderson model in the paramagnetic Replica-Symmetric phase. In two dimensions 
we find that the spurious spin-glass phase transition of the Bethe approximation disappears and the 
' paramagnetic phase is stable down to zero temperature on the square lattice for different random 

f~| , interactions. The quantitative estimates of the free energy and of various other quantities improve 

those of the Bethe approximation. The plaquette approximation fails to predict a second-order spin- 
glass phase transition on the cubic 3D lattice but yields good results in dimension four and higher. 
We provide the physical interpretation of the beliefs in the replica-symmetric phase as disorder 
distributions of the local Hamiltonian. The messages instead do not admit such an interpretation 
' and indeed they cannot be represented as populations in the spin-glass phase at variance with the 

Bethe approximation. 

The approach can be used in principle to study the phase diagram of a wide range of disordered 
' O ' systems and it is also possible that it can be used to get quantitative predictions on single samples. 

' These further developments present however great technical challenges. 
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I. INTRODUCTION 



In the last decade two important results have appeared in the context of Spin-Glass Theory and disordered systems. 
In [l[ the formulation of the Replica-Symmetry-Breaking (RSB) ansatz in terms of populations of fields for the 
\^ Bethe lattice was presented. This has led both to the possibility of obtaining new analytical predictions in the low 
psj ■ temperature phase of the model and to the introduction of the Survey-Propagation (SP) algorithm that has been 
' applied successfully to random fC-SAT instances On the other hand it was recognized that the well-known 

I Belief-Propagation algorithm corresponds to the Bethe approximation [5], and the Generalized Belief Propagation 
' (GBP) algorithm was subsequently introduced in Ref. 0] as a message-passing algorithm to minimize the Kikuchi 
, free energy, a more complex approximation than the Bethe one that goes also under the name of Cluster Variational 
^ • Method (CVM) j8.][52]. 

. ', Since then the idea of merging these two approaches has been around but has not been developed so far, probably 
' due the fact that the standard understanding of the hierarchical ansatz at the Bethe level comes from the Cavity 
J-j \ approximation, while a Cavity-like understanding of more complex Kikuchi approximations is lacking. In this paper 

. . . , we will show that a cavity-like understanding of CVM, although desirable, is not necessary to implement Replica- 
Symmetry (RS) and RSB in the CVM, everything can be worked out using the replica method. 

The main idea is to apply the cluster variational method to the replicated free energy and then to use the RS 
ansatz or the more general Parisi's hierarchical ansatz in order to send the number of replicas n to zero. We will use 
the message passing GBP formulation of CVM representing the messages as populations of populations of local fields 
and obtain a set of equations that represents the generalization of the Survey-Propagation equations. In principle the 
method can be implemented to compute thermodynamic quantities both averaged over different disorder realizations 
and on a single sample. The main difference with the Bethe case is that the populations appear in the equations in 
implicit form and therefore standard iterative population dynamic algorithms cannot be used to determine them. In 
the simplest cases the solution could be obtained iteratively using Fourier transforms. In appendix [C] we will show 
that, in principle, the problem may be solved at any level of RSB using appropriate integral transforms. Nonetheless, 
it turns out to be very hard the actual implementation of this scheme beyond the RS averaged case or IRSB on a 
single sample. Furthermore the application on specific models tells us that the messages should not be represented 
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by populations but rather by functions that are not positive definite and this represents another technical difRculty, 
together with the fact that the equations are written in terms of integrals in many dimensions. In general we expect 
the technical difficulty of the various approximations to grow very rapidly as we increase the size of the maximal CVM 
regions and the number of RSB steps. 

We will present the approach in full generality, i.e. for any CVM approximation and for any number K of RSB 
steps either with or without disorder averaging. The general presentation is somewhat formal and will be postponed 
to appendices O and [B] instead we will initially discuss the application of the method to the classic plaquette approx- 
imation of the averaged free energy of the Edwards- Anderson model in the paramagnetic Replica-Symmetric phase. 
In section HV] we write down the RS message-passing equations for the model and we discuss some of their features, 
in particular the misleading analogy between the GBP equations on a single sample and the RS equations of the 
averaged system. In section |V] we compute the free energy and find that the spurious spin-glass phase transition of 
the Bethe approximation disappears and the paramagnetic phase is thermodynamically stable down to zero temper- 
ature for different kinds of random interactions. We consider also the 2D triangular and hexagonal lattices. In both 
cases the paramagnetic solution yields positive entropy down to zero temperature, however the triangular lattice with 
bimodal interactions has a spurious spin-glass transition at T = 1.0, much smaller than the Bethe approximation 
result T = 2.07. We also show that the quantitative estimates of the free energy and of various other quantities 
improve those of the Bethe approximation both qualitatively and quantitatively. In section IVll we obtain the location 
of a possible second-order spin-glass phase transition studying the Jacobian of the variational equations around the 
paramagnetic solution. This approach predicts no second-order phase transition on the 2D square lattice, in perfect 
agreement with the most reliable numerical studies [1, [13] • Unfortunately it appears that such a good performance in 
2D spoils the 3D result: the plaquette approximation fails to predict a second-order spin-glass phase transition on the 
cubic 3D lattice, which is well seen in numerical studies [ll[. The same plaquette approximation provides very good 
results in dimension four and higher. Although we do not solve the equations in the spin-glass phase the analysis of 
the Jacobian provides important information on the behaviour of the messages below the critical temperature. Most 
importantly we find that the messages should not be represented by populations of fields but rather by functions 
that are not positive definite. This unexpected feature of the actual solutions pushed us to investigate the physical 
meanings of the various objects involved in the computation. In section I VIII we provide the physical interpretation 
of the beliefs in the replica-symmetric phase as the distributions of the local Hamiltonians with respect to different 
realizations of the disorder. The messages instead do not admit such an interpretation and therefore need not to be 
represented as populations in the spin-glass phase at variance with the Bethe approximation. We will also discuss the 
relationship between our approach and the earlier results of the Tohoku group [ia - [l4| . 

The present approach can be used in principle to study the phase diagram of a wide range of disordered systems and 
it is also possible that it can be used to get quantitative predictions on single samples. These further developments 
present however great technical challenges and in the last section of the paper we discuss some of them. 



II. THE REPLICA APPROACH 

The average free energy per spin of a spin-glass system of size N is computed through the replica method as p^ : 

/= lim--l-ln(Z:;) (1) 

n^o fin N 

Where J label different realizations of the disorder and the angular brackets mean average over them. In recent years 
it has been realized that the Replica-Symmetry-Breaking solution can be usefully applied (through the cavity method) 
to a given disorder realization, in the replica framework this corresponds to write the free energy as: 

/j- lim--T^lnZ}' (2) 

n^o pnN 

The above expression is apparently trivial because the replicas are uncorrelated if we do not average over the disorder, 
however in the spin-glass phase the true thermodynamic state is the one in which the distinct replicas are actually 
correlated because of spontaneous RSB. In the replicated Edwards- Anderson model we can define the following 
functional: 

'^^"^ = -;i]^l^Tr(exp(5]/?J,,f].^.^")) = InTrexp ( ^ln(exp/3J^ | , (3) 

such that the free energy is obtained as the n — )■ limit of the above expression. For a single sample the analogous 
function is obtained removing the averages over the couplings J^ . Although in the present paper we will 
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be interested in the n hmit, it is worth noticing that the rephca cluster variational method can provide an 
approximation to the entire function $(n), which is related to free energy fluctuations from sample to sample [l6l[T7|. 
Moreover, at the RS level of approximation, the value max„ $(ri) may provide a better approximation to the typical 
free-energy than $(0) (at the cost of introducing reweighting terms in the RS integral equations). 

In the following we will consider regions of spins r and we will use the definition ipr{crr) — Yii jeri^-'^'P f^'^^a ^i^'j) 
(or ipr{<^r) = Yii jer P'^Tlia ^i^j ^ givcu Sample) to make contact with the notations of Ref. [6]. The difference 
between the two cases is that in the averaged case iprif^r) is homogeneous over space while it fluctuates on a single 
sample. We will refer to the region of two interacting spins as ij. 

In both cases the functional $(n) can be regarded as the equilibrium free energy of a replicated model. The free 
energy can be expressed through a variational principle. The resulting expression involves an energetic and an entropic 
term. The problem is that the latter is usually very difficult to be treated. A standard way to treat it is Kikuchi's 
cluster variational method. In its modern formulation this method consists in writing the entropic term as a sum of 
entropic cumulants and then to truncate this expansion at some order, see [31 for a recent detailed presentation. 

Basically the starting point of the approximation is to chose a set of regions of the graph over which the model is 
defined. These are called the maximal clusters and the entropic expansion is truncated assuming that the cumulants 
of larger regions vanish. In recent years it has been realized that the variational equations can be written in a message 
passing way @ and we will use this formulation in order to extend the CVM to replicated models, either averaged or 
not over the disorder. 



III. CLUSTER VARIATIONAL METHOD AND MESSAGE-PASSING 

In the following we will briefly present the message-passing approach to cluster variational method of Ref. . We 
will use the same notation of Ref. ^ and we refer to it for a more detailed presentation. 

We will call R a set of connected clusters (regions) of nodes (spins), plus their intersections, plus the intersections 
of the intersections and so on. Then Xr is the state (configuration) of nodes in r and br^x^J (the belief) is an 
estimate of the probability of configuration Xr according to the Gibbs measure. Following [6| we will often use 
the notation br omitting the explicit dependence of the beliefs br{xr) on Xr- Then the energy of region v is — 
— ln]^j^ '0jj(a:i, a:j) — In J^,- '0i(a;i) where the products run over all links and nodes (in presence of a field) contained 
in region r. With this definitions, the Kikuchi free energy reads: 

reR \ X,. Xr / 

where the so-called Moebius coefficient Cg is the over-counting number of region s defined by = 1 — J2reA(s) ^r- 
The set A{s) is made of all ancestors of s, i.e. it is the set of all regions that contain s. The condition Cs — 1 holds 
for the largest regions. 

The cluster variational method amounts to extremize the free energy with respect to the beliefs, under the constraint 
that the beliefs are normalized and compatible one with each other in the sense that the belief of a region can be 
obtained marginalizing the belief of any of its ancestors. It is worth noticing that the Kikuchi free energy does not 
provide in general an upper bound on the true free energy of the model. 

The main result of Ref. [6| was to show that the variational equations for the beliefs can be written in a message- 
passing fashion. In order to do this we define for any given region r: i) the set of its ancestors A{r), that is the set of 
regions that contain region r; ii) the set of its parents 'P(r), that is the subset of its ancestors that have no descendant 
that is also an ancestor of r; iii) the set of its descendants X'(r), that is the set of regions contained in region r; iv) 
the set of its children C(r), that is the subset of its descendants that are not contained in a region that is also a 
descendant of r. One introduces message m^s from a region r to any of its children s. The messages can be thought 
of as going from the variable nodes (spins) in r \ s to the variable nodes in s. They dep end on the configuration of Xg 
but for brevity this dependence is omitted. We also need the following definitions |53| : 

• M(r) is defined as the ensemble of connected (parent-child) pairs of regions (r', s') such that r' \ s' is outside r 
while s' coincides either with r or with one of its descendants. 

• M(r) \ M{s) is the ensemble of connected pairs of regions that are in M{r) but not in M{s). 

• M{r, s) is the ensemble of connected pairs of regions such that the parent is a descendant of r and the child is 
either region s or a descendant of s. 
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Although ah these definitions of sets of regions may look abstract and hard to follow, in the next Section we will 
provide immediately an example on the 2D square lattice which should make these definitions clearer. 
With the above definitions it can be shown 01 that the beliefs can be expressed as: 

br = arlpi^r) Y\. ni-r' s' (5) 
r's'eM{r) 

where ar is a normalization constant because they are probability distributions. The messages obey the following 
equations: 

rUrs = ars^1pr\s{Xr) Yl mr"s"/ ^1 

Xr\s r" s" eM(r)\M(s) r's'£M{r,s) 

where ars is some constant (see below) and 4'r\s{xr) is the set of interactions between the nodes of region r without 
considering those that are just in region s, i.e. ipr\sixr) = ipr{xr)/'4'sixs)- 

The constants in eq. ([6]) can be fixed to any positive value, indeed the messages need not to be normalized. In Q 
the constants ars are fixed to 1, while here, for reason of convenience, we work with messages normalized to unity, 
and the ars have to be intended as normalization constants. Two sets of messages obtained solving the equation with 
two different sets of values of the constants ars are simply related by positive multiplicative factors. 

In general the Kikuchi free energy has to be extremized with respect to the beliefs br under the constraint that 
they are compatible, in the sense that the belief of one region marginalized to one of its subregion s has to be equal 
to bs. This is done introducing appropriate Lagrange multipliers. The results quoted above have been obtained 
showing that there exists an equivalent set of constraints between each parent-child couple (r, s) such that imposing 
these constraints through a set of Lagrange multiplier firs, extremizing with respect to the beliefs and identifying 
rurs = expfirs, one immediately gets equation ([5]); equation ([6]) for the messages is obtained imposing the standard 
marginalization conditions for the beliefs. This makes clear why the message m^s as a function of Xs can be normalized 
to any positive constant, indeed this corresponds to a constant shift in the definition of the Lagrange multipliers firs- 

Once the beliefs are obtained the Kikuchi free energy can be computed. However for our purposes we derive another 
expression of the free energy in terms of the messages. To do this we note that if one substitutes the expression for the 
beliefs in the Kikuchi free energy plus the Lagrange multipliers terms one obtains the following variational expression 
for the free energy 



Fr- = - ^ In 



^A{Xr) n ' 

Xr r's'£M(r) 



(7) 



This expression for Fk is stationary with respect to the messages in the sense that the equations © can be also 
obtained extremizing it with respect to the messages 54]. The fact that we can choose any normalization for the 
messages can be also derived noticing that the variational free energy is invariant under a rescaling of the messages 
Indeed the resulting free energy would differ by a term 

Cr + ^ CA In Apr = (8) 

AeA(r)\pUV{p) / 

as can be seen subtracting the two equations Cr — 1 ~ '^AeA{r) '^A and Cp — 1 ~ '^AeA{p) '^^^ ^1- ^'^'^ 0- 
For later convenience we also introduce the following messages normalized to unity: 

rhrs OC nirs Y\_ nir's' (9) 
r's'£M(r,s} 

Accordingly the tilded messages defined above obey an equation with no messages at the denominator: 

rhrs = ars ^ 'ipr\siXr) ]^ mr"s" (10) 

Xr-\a r"s"eM{r)\M{s) 

In the case of replicated models Xr defines the state of the spins in regions r where each spin is replicated n times. 
For any finite integer n the above equations in principle can be solved explicitly, but in order to make the analytical 
continuation to real small n we need to rephrase them in an appropriate way. This will be done using the hierarchical 
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ansatz. The hierarchical ansatz was introduced by G. Parisi in the context of fuhy-connected spin-glass models [15| 
and later extended to spin-glasses defined on random lattices where the Bethe approximation is correct [J-Q- It is 
also called the Replica-Symmetry-Breaking (RSB) ansatz and can be introduced with different levels of RSB steps 
K. The value K = corresponds to the Replica-Symmetric case that in spin-glasses is assumed to be correct in the 
paramagnetic phase valid at high enough temperature or magnetic field. The RS parametrization is already non-trivial 
in the present general CVM context and presents some substantial differences with the Bethe approximation. 

In the following sections we will consider the message passing-formulation of the CVM plaquette approximation at 
the RS level and study its high-temperature phase quantitatively. The general RSB for a generic CVM approximation 
will be presented in the Appendices at the end of the paper. 



IV. THE REPLICA-SYMMETRIC PLAQUETTE APPROXIMATION 



A. Message passing equations 



In this section and in the following we study the plaquette approximation for the replicated Edwards- Anderson 
model on various regular lattices in dimension D. The plaquette approximation is the oldest improvement on the 
Bethe approximation [8] . A detailed presentation of its message-passing formulation can be found in [6] and [1^ . In 
this approximation there are three regions: the plaquette of four spins, the couple of spins and the single spin (the 
point-like region). To make connection with the more abstract definition of regions given in the previous Section, please 
note that ^(point) = {plaquette, couple}, 7'(point) — {couple}, ^(couple) = {plaquette}, ^'(couple) — {plaquette}, 
I?(couple) = {point}, C(couple) = {point}, I?(plaquette) = {couple, point}, C(plaquette) — {couple}, and void sets 
are not reported. Thus we deal with two types of replicated-spins messages, from plaquette to couple and from couple 
to point. If we do not average over the disorder the messages depends on the position of the corresponding regions 
on the lattice while if we average over the disorder all the replicated messages are the same and we have to deal with 
just two of them. 

We will work at the RS level, i.e. with K — RSB steps. Physically this corresponds to the case where on 
each sample there is a single thermodynamic state and the messages are just numbers that fluctuate over space. 
Correspondingly in the averaged case the messages are functions that are spatially homogeneous as we will see below. 
On a single sample the equations we will obtain correspond to GBP, while in the averaged case the equations are new. 
As we will see in the following, looking at the equations of the averaged case as if the message functions represent 
the spatial distribution of the GBP messages on a single-sample is completely wrong; we will come back on this issue 
several times later in the paper and finally present the correct interpretation of the various quantities in section I VIII 
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FIG. 1: A portion of the 2D square lattice. In the plaquette approximation we have couple-to-point messages from, say jg to 
/, and plaquette-to-couple messages from, say, fglm to Im. 



Let US start with the equation for a given sample defined on a 2D square lattice. At the RS level this corresponds 
to work with a single replica. With reference to Fig. [T] and according to eq. ^ we see that we have two types of 
messages. The first type of message is from a couple of spins, say fg, to a spin / and it is a function Pfgj{<7f) of the 
value of the Ising spin cry. As a consequence the message can be parametrized by a single field Uj^ ^ according to the 
following expression: 

Pfgji'^f) = <^Ml3ujgjCrf]/{2 cosh{l3uj ^ j)) , (11) 
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The second type of messages is from a region of four spin, say, fglm to a couple of spins Im and it is a func- 
tion Pfgim,imi<^h<^m) of the two Ising spins {ai,Um), as a consequence it can be parametrized by three fields 
iUfJ^im,im^u^fgim,i-m^ufgi„^i^) in the following way: 

Pfglm,lm{'^l,(^m) = ^'^-'PiPU^fglm^lm^l^m + P^^f glm.lm^l +l^'^Tglm,lm'^m]-^{Ufgi,nijn,U^fgijnijn,U™gi^ij^) (12) 

Where -^{Uf^ij^ imT'^''fgirn irn^''^Tgim im) ^ normalization constant that enforces the condition 

J2{ai ar^} Pf9i"i,imio'i,am) = 1- Now cquatious ([6]) can be rewritten in terms of the various u-fields and yield 
a closed set of equations for them. In order to write down these equations explicitly we introduce the following 
functions: 

unr \ ^ 1 1 cosh P{U + U2) 

h{U, u„ u,) -u, + - In (13) 



and 



where 



UiU,„U,„Uu,u„u„u„u,) = ±ln||i^a_i_l) (14) 
Ul{Ui2,U23,U34,Ui,U2,U3,Ui) = ^ In ^^^^ (15) 

U2{Ul2,U23,U34,Ui,U2,U3,U4) = ^ In (^6) 



K{ai,(T4)= ^ B{ai,(T2,(73,a4) , (17) 

{o"2,0"3} 



B{ai,a2,(J3,0-4) = exp/3[{7i2 0-1172 + U23<J2<J3 + C/34CT30-4 + UiCTi + U2<72 + UgCTa + U4a4] . (18) 

The equation for the field u^^ ^ reads: 

"/9,/ = '^^^bcfg.Jg + ^^fglmjg + Jfg^^bcfgjg + "^fglrnjg^ '^bcfgjg + ^fglnijg + ^cg.^g + ^?ng,g + Kg,g) (19) 

where the l.h.s. corresponds to the r.h.s. of equation © and the r.h.s. corresponds to the r.h.s. of eq. ((TU]). The 
equation for the 2-field from the plaquettc fglm to the couple Im reads: 

U'ZmMn = Um (20) 



fglm,lm+Ufi^l = Ul(#) (21) 

Ufglm.lm + U7m,m = Mif) (22) 

where the notation # stands for the fact that we have to substitute the arguments of the functions U,U2 and U2 
according to: 

Ul2 = f/e^.M/ + (23) 

f^23 = Ui^^^^j^ + Jfg (24) 

U34 = t/^hm„,g„i + Jgm (25) 

ui = 'uifu,if (26) 

"2 = uif^iif + ul^gjg + ui^j + 4^ J (27) 

"3 = ML/g,/g + "ghmn,5m + "?5,g + <g,g (28) 

U4 = u'gh^n^gm (29) 

Again the l.h.s. of eqs. (|20l21l22p correspond to the r.h.s. of eq. ^ while the r.h.s. correspond to the r.h.s. of eq. 
([TU]) . As we already said up to this point we have just written the GBP equations of ^Q]. 
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Now we turn to the replicated CVM and we study it in the average case. On each site, say /, there are n spins 
<Jf = (CTp . . . ,0"") and the rephcated spins interact with their neighbors, say dg with an interaction term of the 
form %pfg{(Tf,(Tg) = J P{Jfg)dJfgexp[l3Jfg'^^^^ ^ajag]. The general RS and RSB ansatz of a function p{(t) of n 
spins was originally presented in [l^ , later its parametrization in terms of distributions of fields was suggested in 
and later revisited in and we refer to those paper for an explanation of the main ideas underlying it. We have 
generalized it to a generic function p(o'i, . . . , CTp) where each ai is a set of n Ising spins, in this section we present the 
RS case while the general RSB case is presented in the appendices. 

Let us consider the first kind of message, the couple-to-point, say P/g,/(o'/). For each integer n we could parametrized 
it through 2" — 1 fields, however such a construction is not suitable to perform the rt — ^ limit. Therefore following 
[H. [19I [20I we parametrized it through a message function qfgj{uj:g j) in the following way: 

Pfaji^f) / dqfgjexp[(3uf^j '^^KS cosh/J^^^^)"" (30) 

Q = l 

where we have used the shorthand notation dqfgj = qfgj{ujg f)'iujg j. The plaquette-to-couple message in turn is 
parametrized through a message function Qfgim,imiU^fgim,im,u^fgim,im^ufgi„^ i„^) as: 

Pfglm,lm{^l,Crm) — J dQfglmJm-f^{U^f^ij^ ij^,u''jgij^ ij^,uJ'gi^^ i^^)" X 

n n n 

a—1 a—1 a—1 

where we have used the shorthand notation 

^Qfglm,lm — Q f glm^lmiU f glra,lmi f glm^lmi f glm,lm}^^ f glrrL,lrrL f glm,,lm f glm.lm 

(32) 

The above parametrization allows to rewrite the message-passing equations ([6|) as a set of equations for the message 
functions for any replica number n. The derivation of these equations is conceptually straightforward and it is given 
in the appendix. Here we just quote the results, in particular in the n — >■ we have: 

= J ^i^^fgj - h)dQbcfgjgdQfgiraJgdqcg,gdqgh,gdqgm,gdP{Jfg) (33) 

where the arguments of the function h are as in eq. ([T9|) . dP{Jfg) = P{Jfg)dJfg and P{Jfg) is the disorder distribution 
of the quenched coupling Jfg. The equation for Qfgim.im is obtained from eqs. ^ and (jlOp . The l.h.s. is given by 
the r.h.s. of eq. ^ and reads: 

11 I s / T tI T tI Tt\ \ f " I I I \ s / " nx Til \ 

l.n.S. = / 0[(Jfgi„^i„^ — jglm,lmr\^fglm,lra " ^fglm,lm ~ '^fl,l)"\^fglm,lm ~ ^fglm,lm ~ ^gm,m) ^ 

X dQfgimdmdqfljdqgm,m ■ (34) 

The r.h.s. is given by the r.h.s. of eq. (|10p and reads: 

r.h.s. = j S{U'/;;i„,^i„, - Um)5{u'fgi^j^ - ui{#mn7gim,ir,. - M*)) x 

X dP{Jlf)dP{J fg)dP{Jgm)dQe fil,fldQl)cfg,fgd'Qghm7i,g7n dqifjdqbfjdqcg,gdqgh,g ■ (35) 

where the arguments of the functions U,U2 and U2 are as in eqs. (|20l21l22p . Now equating the l.h.s. and r.h.s. written 
above for each value of the auxiliary variables (f^/™im,im' ■"/g;m,;m' ^'"gim.im) S^t the equation for Qfgi,n,im- 

The CVM free energy can also be expressed in terms of the various message functions Qfgim,im and qfgj- It is 
presented in full generality in appendix [B] and will not be reported here. The resulting expression is variational in 
the sense that the above equations can be also obtained extremizing it with respect to its arguments i.e. the various 
functions Qfgim.im and qfgj- The number of replicas n appears as a parameter in the variational free energy and the 
analytical continuation to non-integer values can be performed. As we will see in appendix [B] in order to derive such 
an expression it is crucial to start from the variational expression ([7]) which depends on the messages and not on the 
beliefs as expression 
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B. 



Discussion 



In the following we will briefly discuss the equations just obtained for the message functions. First of all we note 
that in the average case the replicated Hamiltonian is spatially homogeneous and therefore it is natural to assume 
that all the couple-to-point message functions are described by a single function q{u) and all the plaquette-to-couple 
message functions are described by a single function Q{U,ui,U2)- On the other hand, the above equations gives us 
an idea of what it would look like to be working on a single sample at the IRSB level. Indeed in this case we should 
have message functions fluctuating in space and obeying the above equations (obviously without the average over the 
couplings J) . As we will see in appendices El and [B] the above equations would corresponds to make the IRSB on a 
single sample with xi — Q while in order to treat the general case a;i > we should add the appropriate reweighting 
terms predicted by eqs. (|B4p and (jBSp . 

We note two important technical difficulties that one has to face solving the above equations, also in the averaged 
case in which one have to deal with just two integral equations for the functions q{u) and Q{U,ui,U2). First of 
all they involve integrals in many dimensions, e.g. eq. (j35l) requires in principle the computation of integrals in a 
16-dimensional space although many of the variables enters the functions U,ui and U2 as sums, see eqs. (|23l - [29| . and 
the actual number of dimensions can be reduced to 7. In this particular case other tricks can be used to reduce the 
number of integrations to 5, but in general we expect that finding the actual solutions of the equations will be a very 
challenging problem. Second and most important we see that the message functions Q{U,ui,U2) and q{u) enter the 
equations in an implicit form and in an iterative scheme one should be able to deconvolve the l.h.s., eq. (1341) . At the 
RS level, or IRSB level on a single sample, this can be done using Fourier transforms. 

Looking at the GBP equations on a single instance eq. and eqs. (|20l21l22p and at the equations in the average 
case for the functions Q{U,ui,U2) and q{u) eqs. (I33I34I35I) one could be tempted to think that the functions q{u) 
and Q{U,ui,U2) represent the distributions over difi^crent disorder realizations of the corresponding GBP messages 
on a given sample. As we will show in sections I VII and IVIIi this interpretation is wrong and misleading. It is wrong 
because it will corresponds to the assumptions that messages passed from the same region are uncorrelated and it 
is misleading because it will lead to the expectation that, being distributions, they are positive definite, which turns 
out to be false. The fact that the message functions are not positive definite in general represents another technical 
difficulty because it means that they cannot be represented as populations, a fact that could have simplified the 
evaluation of the integral equations. 

We note that all these difficulties are absent at the Bethe approximation level. In this case: i) there are no 
convolutions in the equations and ii) the message function itself (and not only the beliefs, see section rVlIj) admits a 
physical interpretation as a distribution, consequently it can be represented by a population and the equations can 
be solved by a population dynamic algorithm. 

In the following we will solve the integral equations in the zero-field paramagnetic phase, where no convolutions 
are needed (the same is not true even in the paramagnetic phase if we consider a maximal cluster larger than the 
plaquette or in presence of a field). Then we will take the first steps into the spin-glass phase, studying the location of 
the critical temperature Tc and finding the message functions at temperatures slightly below Tc. These studies pave 
the way for an analysis deep in the spin-glass phase that is left for future work. 



In the high-temperature region we expect the system to be in a RS paramagnetic phase. We also expect that CVM 
is correct at least at high enough temperature because it amounts to neglect spatial correlations beyond a fixed length 
while the actual correlation length goes to zero at infinite temperature making the approximation more precise at 
higher temperature. In the paramagnetic phase the symmetry breaking fields u vanish meaning that the spins have 
no local magnetization. Thus in this region the variational equations admit a solution of the following kind: 



V. 



THE PARAMAGNETIC PHASE 



A. The square lattice 



q{u) = 6 
Q{U,Ui,U2) = Q 

where Q{U) satisfies the following self-consistency equation 



6{u) 

Q{U)5{ui)5{u2) 



(36) 
(37) 



Q{U) = j ^[U - arctanh(tanh(/3(J7L + Jl)) tanh(/3(L/[/ -I- Ju)) tanh(/3(t/fl, -I- Jr))) //?] 

dP{JL)dP{Ju)dP{jR)dQ{UL)dQ{Uu)dQ{UR) (38) 
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The corresponding variational free energy is given by the following expression 



■ I3F ^ ln(2) 



In 



cosh (/3(J + Ui + U2)) dP{J)dQ{Ui)dQ{U- 



In 



cosh J + [/)) dP{J)dQ{U) 



In 



1 + tanh (/3(Jl + Ul)) tanh {p{Ju + Uu)) tanh {I3{Jr + Un)) tanh {P{Jd + C/d)) 

dP(JL)dP(Jc/)dP(Jfl,)dF(Ji5)dg([/i)dg([/y)dQ([/;j)dQ([/i3) 



(39) 



As we can see the fact that it's vanish simplify considerably the problem because we do not have convolutions to take 
and the functions Q{U) (that in this case can be interpreted as a probability distribution) can be obtained either 
through a population dynamics algorithm or by solving iteratively a discretized version of the variational equation. 
For symmetry reason the solution is such that Q{U) = Q{—U). In spite of its relative simplicity the paramagnetic 
solution in the plaquette approximation yields very interesting results. 

We start considering the 2D Edwards- Anderson model with bimodal interactions J — ±1. Although an analytical 
solution of the 2D Edwards- Anderson model is missing, numerical studies indicate that the critical temperature of the 
model is likely to be zero. Moreover a very recent analytical study by Ohzeki and Nishimori ^2^ finds strong evidences 
for the absence of a finite-temperature spin glass transition in 2D spin glass models. In the following section we will 
study the possibility of a second order spin-glass phase transition looking for a temperature where small non-zero 
fields u develops. 

Before entering into the details we summarize the main results: 

• the paramagnetic phase is thermodynamically stable down to zero temperature, in the sense that it predicts 
always a positive entropy. 

• Q{U) converges to a distribution concentrated on the integers in the zero temperature limit yielding a positive 
zero-temperature entropy. 

• there is no spurious spin-glass phase transition (see next section). 

In fig. [2] wc show Q{U) at temperature T = .1, it is already clear that the solution is converging over the 
integers. At T = the population converges to a distribution concentrated over integers values even if the starting 
point was a distribution concentrated on real values, in particular we have: Q{0) = .534, Q{1) = Q{—1) ~ .226, 
Q(2) = Q(-2) = .006, Q(3) = g(-3) = 0(10-^5). 




FIG. 2: The message function Q{U) of the paramagnetic RS plaquette approximation at T 
integers in the T — >■ limit. 



.1, its support converge on the 



In the case of bimodal distribution of the couplings the fact that the distribution concentrates over integer values is 
usually considered an indication on the quality of the approximation, we will discuss this issue in more depth below 
when studying the behaviour of the specific heat. We anticipate that in 3D the function Q{U) does not converge over 
the integers at low temperatures, thus indicating that the paramagnetic solution is not good down to zero temperature 
in agreement with the expectation that there is a finite temperature spin-glass phase transition in 3-D. 

Furthermore convergence on the integers is necessary in order to recover the expected high degeneracy of the ground 
state energy and correspondingly a non-zero entropy at zero temperature. Indeed the zero temperature entropy can be 
computed either studying the behaviour of the free energy at low temperatures or working directly at zero temperature. 
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The latter approach usually yields more precise estimates and we have followed it to compute the zero-temperature 
entropies reported below. To work at zero temperature one needs to consider the so-called evanescent fields writing 
U = k + eT, where k is an integer and e is a real number describing the deviations from the integers at small finite 
temperature. Then the function Q{U) is replaced by a function Q(k,e) and both the zero temperature energy and 
entropy can be expressed in terms of Q{k,e). When there is no convergence over the integers one can in principle 
study the solution that is obtained considering only integers values (the equations are stable on the integers). In this 
case however the lack of convergence problems shows up when one tries to compute the zero temperature entropy, 
because the evanescent fields e diverges and have no stable distribution. 

In fig. [3] we plot the CVM free energy as a function of the temperature. At zero temperature we find Eq = —1.43404 
that has to be compared with the best numerical estimate Eq — —1.401938(2) [23l. |24| . The Bethe approximation 
result is instead Eq = —1.472(1) and Sq = 0.0381(15) (from a numerical study on the Bethe lattice [23). Note that 
the Bethe approximations predicts also a spurious spin-glass phase transition at a temperature T = 1.51865. Thus 
we see that the estimate of the ground state energy is better than that of the Bethe approximation. Nevertheless the 
estimated zero temperature entropy Sq = 0.010(1) is too low, compared to the expected value = 0.0714(2) [26Ll27l|. 
the reason for this is evident from the figure: the quality of the CVM approximation decreases approaching T — 
where the correlation length of the actual model is expected to diverge. 

We recall that the Bethe approximation has the peculiar property that it is correct on the Bethe lattice. Therefore 
there always exists a thermodynamically stable solution in the Bethe approximation, i.e. the one that yields the free 
energy of the Bethe lattice. This is not true for a general CVM approximation and it is the so-called realizability 
problem in CVM theory [3]. However it was noted in [23] that on some models the plaquette approximation yields 
the exact result for the free energy. 



'1.9- / 

0.5 1 1.5 



FIG. 3: Free energy vs. Temperature of the paramagnetic solution in the plaquette approximation (solid) for the 2DEA model 
with bimodal couplings. The paramagnetic Bethe solution (dotted) is unstable below T — 1.5186 (dot), the model on the Bethe 
lattice has a spin-glass phase transition at this temperature. The straight lines are Eo — TSo where Eo (So) is the ground state 
energy (entropy) for the true model (dashed) and for the Bethe lattice (solid) from numerics. 



Gaussian Distributed Couplings - We considered also the 2DEA model with Gaussian distribution of the couplings. 
Again we find that the paramagnetic phase is thermodynamically stable down to zero temperature where it predicts a 
vanishing entropy according to what is expected. In this case the CVM estimates are even better than in the previous 
case. In fig. |3]we plot the free energy as a function of the temperature, the ground state energy reads Eq = —1.3210(2) 
to be compared with the numerical prediction Eq — —1.31479(2) p4| . 



B. Triangular and Hexagonal Lattices 



We studied the spin-glass with bimodal distribution of the couplings defined on the triangular lattice and on 
the hexagonal (a.k.a. honeycomb, brickwork) lattice, using respectively the triangle and the hexagon as the basic 
plaquette, see fig. [5l Much as in the square lattice case, the messages are parametrized by a single function Q(U) in 
the RS paramagnetic phase representing respectively the triangle-to-couple and hexagon-to-couple messages. In the 
average case, self-consistency equations for the messages are exactly as Eq. [38l with the only difference being in the 
number of hyperbolic tangents contained in the argument of arctanh; in other words messages satisfy the following 
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FIG. 4: Free energy vs. Temperature of the paramagnetic solution in the plaquette approximation (solid) for the 2DEA model 
with Gaussian couplings. The dot at T = corresponds to the value of the ground state energy of the actual model. 



equations in distribution sense 

U = arctanh(tanh(;3([/i + Ji))tanh(/3(J72 + J2)))//3 (40) 

U = arctanh(tanh(/?(;7i + Ji)) tanh(/3(C/2 + J2)) tanh(/3(C/3 + J3)) tanh(/3(C/4 + J4)) tanh(/3([/5 + J5))) //3 (41) 

for the triangular and hexagonal lattice, respectively. In both cases we found again that the paramagnetic phase is 
thermodynamically consistent down to zero temperature in the sense that the entropy of the paramagnetic solution 
is always positive. The function Q(U) converges on the integers for the triangular and on the half-integers for the 
honeycomb lattice predicting a non-vanishing entropy at zero temperature in both cases. 




FIG. 5: We have considered the spin-glass model with bimodal couplings on the three regular 2D lattices using respectively 
the hexagon, square and triangle approximation 

In fig. |6] we plot the free energy of the triangular lattice as a function of the temperature. The most accurate 
predictions for the ground state energy and entropy come from a Pfafhan method [2^, giving Eq = —1.7085(1) and 
5o = 0.065035(2). The Bethe approximation predictions 2^ are Eq = -1.826(1) and So = 0.0291(10) while the 
present CVM triangle predictions are Eq = —1.74227 and Sq = 0.0087(7). We see that much as in the above cases the 
CVM estimate of the free energy largely improves upon the Bethe one. Note that the Bethe approximation predicts 
a spurious low-temperature spin-glass phase that appears at T = 2.078086. The analysis of the next section shows 
that the triangle approximation predicts a spurious spin- glass phase transition at T = 1.0. A detailed study of the 
spin-glass solution goes beyond the scope of this work but we expect that it will improve the estimate of the ground 
state energy and entropy. As in the square lattice case, the zero temperature entropy is less precise than the energy 
for the same reasons discussed above. 

In fig. [7]we plot the free energy of the hexagonal lattice as a function of the temperature. The numerical predictions 
for the ground state energy and entropy are respectively Eq ~ —1.2403(2) and Sq = 0.02827(5) ^3d\. The corresponding 
Bethe lattice predictions [25] are Eq = —1.2716(1) and Sq — 0.0102(10) while the present hexagonal CVM predicts 
Eq = —1.242187 and Sq — 0.020. In this case also the zero temperature entropy improves over the Bethe result. In 
the case of the hexagonal lattice both the Bethe and CVM approximations are much more precise than on the square 
lattice, and the CVM approximation corrects 95% of the error of the Bethe approximation. Note that the Bethe 
approximation predicts once more a spurious low temperature spin-glass phase that appears at T = 1.13459. We have 
not investigated the presence of a spurious low temperature spin glass phase in the hexagonal CVM approximation, 
since this study is computationally heavy and we strongly expect such a phase not to exist. Indeed the hexagonal CVM 
approximation is more accurate than the square CVM approximation, and the latter does not shows any spurious 
spin-glass phase at low temperatures. 
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FIG. 6: Free energy vs. Temperature of the paramagnetic solution in the triangle approximation (solid) for the 2DEA model 
with bimodal couplings on the triangular lattice. The entropy is positive down to zero temperature, however the dot on the 
curve marks the temperature where a spin-glass solution should be found. The paramagnetic Bethe solution (dotted) is unstable 
below T — 2.0780869 (dot), the model on the Bethe lattice has a spin-glass phase transition at this temperature. The straight 
lines are Eo — TSo where Eq (So) is the ground state energy (entropy) for the true model (dashed) and for the Bethe lattice 
(solid) from numerics [25j . 
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FIG. 7: Free energy vs. Temperature of the paramagnetic solution in the hexagon approximation (solid) for the 2DEA model 
with bimodal couplings on the Hexagonal lattice. The dot at T = represents the ground state energy of the actual model. 
The paramagnetic Bethe solution (dotted) is unstable below T — 1.13459 (dot), the model on the Bethe lattice has a spin-glass 
phase transition at this temperature. The straight line is -Eo — TSo where Eo (So) is numerical estimate for the ground state 
energy (entropy) for the Bethe lattice (solid). 



In the 2D square lattice spin-glass models we have found that the paramagnetic phase is thermodynamically stable 
down to zero temperature. This is in agreement with numerical evidence that show that the only critical point is 
at T = 0. Therefore it is natural to ask whether the CVM approximation also predicts a zero-temperature critical 
point. One can argue that this is not the case. Indeed both for models with Gaussian and bimodal interactions it 
can be seen that a magnetic field scaling with the temperature a.s H = hT with h small has an effect 0{T) on the 
free energy, therefore the derivatives with respect to the field diverge as d^F/dH^ = 0{T^~^). On the other hand 
the fluctuation-Dissipation- Theorem tells us that the quartic derivative is related to the fluctuations of the overlap 
[i.e. the spin-glass susceptibility) times T^^. Thus we see that a quartic derivative diverging as does not imply 
a divergent spin-glass susceptibility at T = 0. 

C. Free energy fluctuations 

In random systems the free energy fluctuates from sample to sample. The scaling of the variance with the system 
size is rather nontrivial in mean-field spin-glass models like the Sherrington-Kirkpatrick model [l6l Isil . [3^ and in 
random graphs with fixed connectivity with bimodal interactions pll . l33j |. On the other hand in finite dimensional 
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systems it is known that the variance must scale as the square root of the volume [33, |3J| . In particular at system 
size N the fluctuations A/j = fj — (fj) of the free energy fj of a given sample around its average value obey: 

(A/j) - ^ (42) 

Although the mean-field prediction is u = it has been shown that the loop corrections leads to a non-zero a [ssl [36| . 
The present approach also predicts a non-zero a and it allows to get a quantitative estimate. 

In the replica framework it can be shown that a is related to the 0{n^) term in the expansion of n$(n) at small 
n (see [iBl)- It has been recently noted [2l| that on the Bethe lattice one can compute the 0{n^) expanding the 
variational expression of n^{n) around n = 0. Since the expression is variational the total second derivative with 
respect to n coincides with the partial derivatives evaluated at n = 0. The same argument applies in any CVM 
approximation. In order to obtain the results we need the variational expression of the CVM free energy written in 
terms of the message functions that it is presented in full generality in appendix [B] the following discussion is based 
therefore on the definitions and results of appendix and |B] One has to expand the expression of the variational 
free energy eq. (jB13l) in powers of n at the second order and evaluate it using the Q{U) corresponding to n = 0. We 
immediately see that the 0{n^) is given by the sum over the different regions r of the fiuctuations of the corresponding 
free energy with the usual region coefficients Cr- In particular we define the free-energy variance of region r at generic 
number of RSB steps K as: 



^ (((A<-^V)) - ((AFj^-^^))^ (43) 



where we have used the definitions: 



m)= n P^^J"dP^^r^s^'\0)j (44) 

r"s"eJ\/(r) 

We recall that the RS approximation corresponds to i^T = 0. With the previous definitions we have: 

As usual if we assume that the distribution of the couplings P{J) is the same for all Jij we can conclude that the 
contributions of regions with the same form are equal and replace the above expression with a sum over types of region 
(e.g. the plaquette, the couple and the point in the plaquette approximation) each multiplied by the number of regions 
of a given type per spin, see discussion below eq. (jB13|) . Note that the Cr can be negative and therefore a wrong 
approximation could yield a negative tr^, for instance a negative ct^ is predicted by the RS solution of the SK model 
below the critical temperature [3l|, in agreement with the fact that the correct solution is not RS. On the contrary 
in all the 2D models considered we have found that the paramagnetic RS solution of the CVM approximation yields 
a positive down to zero temperature in agreement with the expectation that the actual model is paramagnetic at 
any finite temperature. 

For completeness we report the expression of the free energy fluctuations cTsquare of the plaquette approximations 
for the square lattice 2D model in the paramagnetic RS approximation. According to the above equations we have: 

^square ^plaquette couple ^~ ^point (46) 

In the paramagnetic approximation all the symmetry-breaking small fields vanish, i.e. q{u) — 5(u) and Q(U, ui, U2) = 
Q{U)5{ui)5{u2) therefore (Jpoint — 0- For the couple we have: 

Ai^Lpl(C/c/, Ud, J) ^ i \ncoshp{Uu + Ud + J) (47) 
where we have neglected unimportant constant factors. The contribution of the couple reads: 

<yloupie = / ^ cos}vP{Uu+UD+J)dQ{Uu)dQ(UD)dP{J)- Q j In cosh /?([/[/ + Ud + J)dQ{Uu)dQ{UD)dP{J)^ 

(48) 

For the plaquette we have: 

^Filaquettei*) = ^ ^ exp/3((t/l5 + Jd)(J1<J2 + [U L + JL)fT2C73 + {UlJ + J[/)a3tT4 + [U R + Jfl)aif74) (49) 

(7\ ,cr2 ,(73 ,(74 
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where the argument # stands for {Ud, Ul, Ujj, Ur, Jd, Jl, Ju, Jr) and: 

^llaguette = / ^F^uZettei*? dQ{U D)dQ{U L)dQ{Uu)dQ{U R)dP {J D)dP(J L^^^^ 

^F^7alieUei*)dQ{UD)dQ{UL)dQ{Uu)dQ{UR)dP{JD)dP{JL^^^ (50) 



In particular at zero temperature we have a'^g^are — 0.1678, af^^^ = 0.100 and cr^^j = 0.373 respectively for the 
square, hexagon and triangle CVM approximation of the corresponding 2D lattices with bimodal coupling. In the 
case of the square lattice with Gaussian couplings we estimate ct^ = .536 leading to ct = .732 in very good agreement 
with the value a = .725 for the actual model reported in [33l |. 

We note that the estimates for the 2D lattices with bimodal interactions represent a critical improvement with 
respect to the Bethe lattice where one has (7 = because of the spatial homogeneity of the model [2l|, [sl] . 



D. Specific heat at T = 

Another interesting prediction of the CVM approach regards the behaviour of the specific-heat at low temperatures. 
In the case of the square lattice it was suggested long ago by Swendsen and Wang [33| that the behaviour of the specific 
heat at low temperature is of the form 

cv - ^aexp[~A/T] (51) 

with A = 2. This is absolutely non-trivial because the energy of any finite-size excitation for the square lattice is a 
multiple of 4, and this would lead instead to ^ = 4 as later claimed in Over the years the A = 2 result has been 
supported by many authors [27l l39l - l4li] ; recently another scenario has also been proposed in which cy behaves as a 
power law [lO| with a universal exponent that is the same of models with Gaussian distributions of the couplings. 
The true nature of cy remains nevertheless unclear . 

Given that Q{U) is symmetric it follows that in both the square, hexagonal (in agreement with [4^) and triangular 
lattice the behaviour of the specific heat at low temperature predicted by the CVM approximation is of the form 
([5T|) with A = 2 and p = 2. Note that this is a non-trivial prediction not only for the square lattice but also for the 
triangular one. It must be remembered however that the CVM is intrinsically a mean-field approximation and could 
never give a power-law behaviour. On the other hand it clearly suggests that even if a power-law behavior is actually 
present there are corrections of the form (|5ip with the non-trivial value A = 2. 

The CVM approach yields also the numerical coefficient a of the leading term in ((FT|) that can be computed 
working directly at zero temperature. Given that the correction is exceedingly small it is safer to work directly at 
zero temperature. In order to compute this coefficient one has to write the finite-temperature field U as 

U = k + Te + Te^^^^z (52) 

and study the joint distribution of the triplets Q{U) — > Q{k,e^z) whose equation was obtained considering the 
leading order contribution of the equation of Q{U). Summarizing the low temperature behaviour of the specific heat 
according to the CVM approximation for the 2D Ising spin-glass with bimodal interactions is 

cy = ^exph2/T] (53) 

The coefficient in the case of the square lattice is a « 60 and does not appear to compare well with the numerical 
data, if we go back to fig. |3]we can argue that the error of the CVM square approximation with respect to the actual 
model is not small enough to reproduce quantitatively the behaviour of the specific heat 

At last, it is also interesting to note that much as in the case of O(n^) term discussed above, the replica CVM predicts 
a qualitatively different behaviour than the Bethe approximation. The low-temperature specific-heat behaviour has 
not been considered in the case of the Bethe lattice; nevertheless it is known that the Bethe approximation yields a 
spurious phase transition with non-zero symmetry-breaking fields u concentrated over the integers. In the case of odd 
connectivities (corresponding e.g. to the hexagonal lattice) this leads naturally to a A = 2 gap, instead in the case of 
even connectivities (corresponding to the square and triangular lattices) the fields are known to concentrate over odd 
integers values leading to A — 4, while as we saw the use of the CVM approximation reduces the gap and leads to 
A ~ 2 for all the three 2D model considered. 
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VI. THE SPIN-GLASS PHASE TRANSITION IN THE PLAQUETTE APPROXIMATION 

A typical application of the CVM 8] is the location of the critical temperature of phase transitions. We recall that 
in the plaquette approximation above the critical temperature the small fields u vanish, i.e. we have: 

q{u) = 5{u) (54) 
Q(t/,ui,w2) = Q{U)5{ui)5{u2) (55) 

A spin-glass phase transition corresponds to the fact that the symmetry-breaking fields u become non-zero. Near the 
critical temperature of a second-order phase transition the symmetry-breaking fields u will be no longer zero but small, 
and we will determine the location of the critical temperature considering the second moments of the distributions. 
We define: 



q{u)u du (56) 
ao(t/) := j j Q{U,ui,U2)duidu2 (57) 
aij{U) :— / Q{U,ui,U2)uiUjduidu2 — 1,2 (58) 



For symmetry reason we expect Q(U,ui,U2) ~ Q{—U,—ui,U2) — Q{U, —ui, —U2) = Q{U,U2,ui) thus aii([/) = 
o-22{U), furthermore ao(C/) and aii(C/) = a22(U) will be even function of U , while ai2{U) will be an odd function of 
U. 

Now to explain the basics of the method we consider the simplest case of the Bethe approximation in which only 
the function q{u) is present. In this case the transition is marked by the fact that the parameter a defined above 
vanishes above the critical temperature while it is different from zero below. To determine the critical temperature 
one expands the iterative equation at small a and obtain something of the form: 

C{T)a + B{T)a'^ +0{a^) = Q , (59) 

the critical temperature corresponds to the vanishing of the coefficient C{T), or equivalently to the fact that the 
homogeneous linear equation aC(T) = admits a non-zero solution. The condition C(T) = leads to the equation 
(tanh^ l3cJ)j = 1/c in the Bethe approximation where c -I- 1 is the connectivity of the model. We note that below the 
critical temperature the function q{u) is also described by higher order moments, however to determine exactly the 
critical temperature it is sufficient to consider the behavior of the second moment a. 

We have obtained the corresponding linear homogeneous equation for the variables {a, aii{U) , ai2{U)} in the form: 



a = Ka^aa + J dU'Ka,a,AU')aii{U')+ J dU'Ka^a,AU')ai2{U') (60) 
aao{U)+aiAU) = Ka,,AU)a + J dU' Ka,,^a^^{U,U')aii{U') + J dU' Ka,,,aAU,U')a^2{U') (61) 
ai2{U) - Ka,,^a{U)a+ f dU' K,,,^a^^{U,U')an{U') + f dU' Ka,,,a,MU')a,2{U') (62) 



where the various coefficients K depends on the temperature and on the corresponding function Q{U), we do not 
report them all but in the following we will explain how they have to be obtained. The critical temperature should be 
identified with the point where the above homogeneous set of equations admits a non-zero solution for the parameters 
(a, aii(J7), ai2(J7)). The corresponding eigenvector determines the behavior of the function q{u) and Q{U,ui,U2) 
slightly below the critical temperature. 

To see how the coefficients K have to be obtained we consider the equation for the parameter a. We start from the 
equation for q{u), eq. ()19|) . we multiply both sides times and integrate over u. We have: 

a = / /.^rfQ.e/,/,rfQ/,.™,/,rf.e,„rf.,.„rf.,™„rfP( J/,) (63) 

Where the function h is defined in eq. (|19p . Now we expand the function h at the second order in powers of the small 
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fields u, and express the integrals in eq. (|63p in terms of a, aoiU) and aij(U), the result is: 

a = 3a J P{J)ao{U^)ao{U^) tanh^ I3{J + U° + U'") dJ dU" dU^ + 2 J aii{U)dU + (64) 

+ 2J P{J)ao{U^)an{U") tanh:^ I3{J + + U^)dJdU^ dU^ + (65) 

+ 4 / P(J)ao(C/^)ai2(C/^)tanh/3(J + C/^ + C/^)dJdC/^dC/^ (66) 



The above equation corresponds to eq. (|60|) . The other coefficients can be obtained similarly multiplying both sides 
of the equation for Q{U,ui,U2) [i.e. eqs. dM]) and ([35])) times (Ugfi^^ i^^)'^ and UgfiTn,im^'^fiTn,iTn ^^'^ integrating over. 
The resulting expressions are fairly complicated and we do not write them down here. 

A complete analytical treatment of the problem seems unfeasible, also because we do not have an analytical 
expression of Q{U) at all temperatures. Thus we have discretized the space of the U , assuming that it can takes only 
a finite number 2I„iax + 1 of values in the interval {—Umax, Umax), i-e. U — idu with i = 0, ±1, ±2, . . . , ±Imax and 
du — Umax/ Imax- Correspondingly we have a set of ilmax + 3 variables a = (a, aii{U),ai2{U)). At any temperature 
we first compute Q{U) on the discretized range of U and then we rewrite the set of equation (|60|61l62p in the form 
J • a = where J is a {4,Imax + 3) x {'ilmax + 3) matrix, (in the following we will refer to it as the Jacobian matrix). 

The computation of the coefficients is the technical bottleneck of the computation, we have worked typically with 
Umax = 2 and Imax = 40. The Jacobian matrix is diagonalized and the critical temperature have to be identified 
with the point where it has a vanishing eigenvalue or correspondingly a vanishing determinant. Note that we are 
linearizing the equations and therefore we call Jacobian the matrix that we compute, as a consequence this matrix 
is not symmetric and can have complex eigenvalues. A symmetric matrix would be obtained had we considered the 
Hessian of the free energy. However as far as the determination of the critical temperature is concerned the two 
approaches are completely equivalent. 

We have considered the 2DEA Ising model with couplings J = ±1 on the square lattice. Interestingly enough the 
plaquette approximation predicts no spurious spin-glass phase transition at any finite temperature. In other words 
the determinant of the Jacobian remains always finite, this has to be compared with the Bethe approximation that 
yields a spurious spin-glass transition at T = 1.51865 for the 2DEA model with J — ±1. In fig. [5]we plot the inverse 
of the logarithm of the determinant of the Jacobian at low temperatures. This was obtained using Umax = 2.1 and 
l7nax — 42 therefore the Jacobian is a 171 x 171 matrix. The plot shows that the determinant does not vanish at least 
down to T = .05 and suggest that it does not vanish at any finite temperature. A careful study of its behavior as 
T — > goes beyond the scope of this work. 

A similar study on the 2DEA Ising model with couplings J = ±1 on the triangular lattice shows that instead the 
Jacobian vanishes at T = 1.0 that improves considerably on the Bethe lattice estimate T = 2.078. It is interesting 
to note that also in the 2D triangular antiferromagnet (which has again a zero temperature critical point) the CVM 
approximations yields a spurious phase transition 44] . 
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FIG. 8: Plot of the inverse of the logarithm of the determinant of the Jacobian vs. Temperature for the 2D square lattice with 
bimodal interactions (see text). It is strictly positive down to zero temperature thus there is no second-order spin-glass phase 
transition in the model at finite temperature. 
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A. General dimension 

The plaquette approximation can be also applied to regular lattices in any number of dimensions. The objects to 
be considered are still the messages functions q{u) and Q{U, ui, U2) but the coefficients Cj. of the regions and the total 
number of regions changes. For instance in 3-D we have Cpiaquctto = 1, Ccoupic = —3 and Cpoint = 7 while the number 
of regions per spin are ripiaqucttc — 3, ricoupio — 3, npoint = 1- The total number of messages entering in a given region 
also changes, in particular in generic dimension D there are 2D messages q{u) entering on the point; on the couple of 
points there are 2D — 2 messages Q{U, wi, U2) and 2D — 1 messages q{u) for each point, while on the plaquette there 
are 2D — 3 messages Q{U,ui,U2) on each link and 2D — 2 messages q{u) on each point. The above formalism for 
the study of a second-order phase transition can be extended straightforwardly to general dimension provided some 
care is taken in order for the computation of the Jacobian matrix to be done in reasonable time. In practice we have 
introduced auxiliary functions to represents the convolutions of Q{U) with itself in order that the integrals needed to 
compute the elements of the Jacobian remain three-dimensional as in 2D. 

In dimension higher than two the EA model is largely believed to display a second-order spin-glass phase transition. 
In fig. [5] we plot the value of the smallest eigenvalue of the Jacobian matrix of the 3DEA Ising model. We see that 
unfortunately it does not vanish at all, although it decreases considerably around the temperature where the actual 
model is believed to have a phase transition, T ^ 1.1. The plaquette approximation leads to a disaster in 3D: if we did 
not know the actual behavior of the model we could wrongly think that much as in 2D the paramagnetic phase is stable 
down to zero temperature; however a clear hint that this cannot be the case comes from the study of the free energy. 
In fig. [10] we plot the free energy as a function of the temperature, this shows that the entropy remains positive but 
the free energy has the wrong convexity at low temperature and negative specific heat. Another indication that the 
paramagnetic solution is wrong in 3D at low temperature comes from the fact that at zero temperature the solution 
does not converge on integers values, at variance with the 2D case studied in the previous section. We have also 
considered different distributions of the couplings (Gaussian and Diluted) and check that unfortunately the plaquette 
approximation still does not predict any phase transition in 3D. The present approach is able to detect a second-order 
phase transition where the variables (a, aniU), 012 (C/)) are small, it is also possible that the plaquette approximation 
makes the transition first-order but we leave the investigation of this point for future work. Note that the smallest 
eigenvalue gets very near to zero therefore we expect that in an approximation with a basic region slightly larger than 
the plaquette we should be able to recover the expected phase transition. 
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FIG. 9: Plot of the smallest eigenvalue of the Jacobian around the expected critical temperature T = 1.1 for the plaquette 
approximation of the 3DEA model with bimodal interactions. Since it does not vanish the CVM predicts no second-order phase 
transition in this approximation. 

Fortunately enough, the situation gets better in dimension four (see fig. [TT|) where the smallest eigenvalues vanishes 
at r = 2.2, thus correcting around 2/3 of the error of the Bethe estimate T = 2.51 of the actual value of the 
critical temperature T = 2.03 estimated numerically [45| . Increasing the dimension the quality of the results improves 
systematically, in 5D we have T = 2.550 to be compared with high-temperature series estimates T — 2.57(1) my\ and 
T — 2.54(3) [431, thus correcting almost all the error of the Bethe approximation estimate T = 2.88. 

It is interesting to observe that according to fig. (|lip the smallest eigenvalue in dimension four of the paramagnetic 
solution vanishes again lowering the temperature below T — 2.2 at around T = 1.9, we will discuss this unexpected 
feature of the solution at the end of the present section. 

We have seen that the plaquette approximation gives good results in 2D and in general dimension greater than 
three, while in 3D it leads to a disaster at low temperature. We note that the Bethe approximation is correct at 
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FIG. 10: Free energy vs. Temperature of the paramagnetic RS solution of 3DEA model in the plaquette approximation, although 
in this approximation there is no second-order phase transition, the paramagnetic solution is unphysical at low temperature 
predicting a negative specific heat. 




FIG. 11: The smallest eigenvalue of the Jacobian vs. Temperature for the plaquette approximation of the 4DEA model with 
bimodal interactions. It vanishes at T = 2.2 thus correcting 2/3 of the error of the Bethe approximation T = 2.52 on the actual 
value of the critical temperature T — 2.03 45]. Note the presence of second zero at T ~ 1.9 that we interprete as a sign of 
RSB, see text. 



D = 1 and at D = oo, thus it is natural that the plaquette approximation is a perturbative correction to Bethe in 
high enough dimension as can be seen already in dimension five. Looking at the behavior of the smallest eigenvalue 
of the Jacobian matrix in Z? = 3 we see that, although it does not vanish, it has a minimum around the true critical 
temperature T = 1.1 and it is likely that the second-order phase transition reappears considering a basic region larger 
than the plaquette, e.g. the cube. We note hovifever that this approximation requires to deal with equations for the 
order parameters involving convolutions even in the RS paramagnetic phase, a technical difhculty that is not present 
for the plaquette. 

The study of the spin-glass phase requires to deal with non-zero fields (w, ui, U2) and to deal with the convolutions 
appearing in eq. (|34p . Thus the study of the spin-glass phase requires to tackle this technical difficulty and goes 
beyond the scope of this work. Nevertheless the study of the Jacobian gives us a crucial information on the spin-glass 
phase that we will discuss in the following. 

Slightly below the critical temperature in Z? > 3 the quantities (a, aii([/), ai2(U)) are proportional to the vanishing 
eigenvector of the Jacobian matrix. In other words we have: 

(a,an(t/),ai2([/))=6(T,-r)(Ai«),A£)(C/),AW(C/)) (67) 

where (Ai°^ , Ai° ^ (C/) , Ai°2 (U)) are the components of the eigenvector corresponding to the zero eigenvalue and b is some 
numerical constant that cannot be determined solely from the knowledge of the Jacobian but needs the computation 
of the quadratic terms. Indeed the determination of the proportionality factor requires to include the next order 
terms analogous to the term B{T) in eq. (15^ at the Bethe level. In fig. [T2]we plot the function aii{U) (modulo an 
unknown positive constant scaling as Tc — T, i. e. the non-normalized eigenvector) at a generic temperature slightly 
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below Tc = 2.2 for the 4DEA with bimodal interactions, the proportionahty factor is such that the a component is 
positive, a = 0.08490. 
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FIG. 12: The function an (17) (modulo an unknown positive constant scaling as Tc — T) at a generic temperature slightly below 
Tc = 2.2 for the 4DEA with bimodal interactions, see text. It is negative for some values of U meaning that the function 
Q{U,ui,U2) is not positive definite. 



We see that aii{U) is negative for some values of U and this is puzzling, indeed we recall that the definition of 
aii{U) is: 

aii(C/) = j Q{U,ui,U2)u\duidu2 (68) 

thus if aii{U) is negative below the critical temperature for some U it follows that the message function Q{U, ui, U2) 
cannot be positive definite! The first consequence of this fact is that the function Q{U,ui,U2) cannot be interpreted 
as a distribution function of the messages ([/, ui, U2) on a given sample: had we followed that interpretation we should 
have concluded that the whole approach is inconsistent. In the next section we will discuss this issue in more depth 
and see that instead it is the naive interpretation that is actually inconsistent, in particular we will show that the 
message functions Q{U, ui, -^2) need not to be positive definite while the beliefs of the regions do. 

We mention that negative aii{U) are found also if we study the response of the system to the presence of a small 
field H in the high-temperature phase. In this case we find non-zero values of (a, aii(C/), ai2{U)) of order 0{H^) that 
can be determined inverting the Jacobian matrix and applying it to the 0{H^) perturbation, and again we find that 
while a is positive aii{U) is negative for some values of U. This effect survives in the infinite temperature limit. In 
this regime we find that at leading order the variables to be considered are a and an = / aii(C/) dU and an explicit 
computation shows that a ~ and an ~ —3/3^H^ in any dimension. 

We note that the fact that the messages are not positive definite means that they cannot be simply represented as 
populations and this, together with the presence of convolutions in the variational equations, is a technical challenge 
to be overcome in order to obtain quantitative results for general CVM approximations and for all regions of the 
phase diagram. 

We also mention that the Jacobian approach presented here can be also applied to study the phase diagram of 
models with ferromagnetically biased interactions, in this case one would be interest in the location of the ferromagnetic 
transition and the variables to be used should be a = / q{u)udu and di{U) = J Q{U,ui,U2)uiduidu2dU . 

We end this section with a comment on the peculiar feature displayed by the smallest eigenvalue of the Jacobian 
in four dimension according to fig. (fTTj) . Below T = 2.2 we expect to find a RS spin-glass solution with a non-zero 
positive value of a oc (Tc — T) . As we already said the actual value of the proportionality factor cannot be determined 
solely from the knowledge of the Jacobian but needs the computation of the quadratic terms. However if we assume 
that these terms do not change too much with the temperature between the T = 2.2 and T = 1.9 {i.e. where the 
eigenvalue vanishes again) we easily see that the parameter a should have the opposite behaviour of the smallest 
eigenvalue. In particular, lowering the temperature, it will initially increase from zero to some maximum and then 
decrease again to zero. This would be completely unphysical and means that probably to RS spin-glass solution 
becomes meaningless and has to be abandoned below some temperature greater that T — 2.05 where the smallest 
eigenvalue reaches its minimum. It is tempting to interprete this fact as an indication that the RS spin-glass solution is 
physically wrong at low temperature and that a RSB solution has to be considered instead. One can further speculate 
that if this is the case two phenomena should be observed. First the effect should become more pronounced while 
increasing the size of the basic CVM region. In other words increasing the precision of the CVM approximation the 
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region of validity of the RS spin-glass solution should shrink to zero i.e. the first two zeroes of the smallest eigenvalue 
should tend to coincide. Second for a given CVM approximation the use of a RSB solution should increase the range 
of validity of the solution shifting the point where the solution becomes unphysical to lower temperatures. We think 
that this is a very interesting open problem. 



VII. PHYSICAL INTERPRETATION OF THE BELIEFS 



In the previous section we have seen that the messages functions Q{U, ui, U2) of the plaquette CVM approximation 
in general dimension are not definite positive and cannot be interpreted as distribution functions. In this section we 
will obtain the physical interpretations of the beliefs of br{cr) of the regions of the replicated model. In particular we 
will show that the beliefs in the RS approximation can be interpreted as distributions over the disorder of the local 
Hamiltonians. 

We consider first the belief of a point on the lattice, say 0. In the n-replicated system there are n spins a^, . . . ,(Tq 
on that point. As we saw in section |lT] averaging over the disorder couples the different replicas and the effective 
Hamiltonian becomes — ^ij^i^{Gxp[l3JJ2a'^i'^j])- The belief b{ao) describes the marginal distribution of the repli- 
cated spins at point with respect to the replica Hamiltonian. Any correlation between any number p of the n spins 
at site can be expressed in terms of 6(cto) 

(K^---^o^))=EW---^o')^M (69) 

Co 

where ((•••)) means average with respect to the replicated Hamiltonian. On the other hand such a correlation can 
be written as: 

Where (• • • ) means average over the disorder, Zj is the partition function of the non-replicated system for a given 
realization of the disorder J, (• ■ • ) j means thermodynamic average at given disorder J and too,j is the magnetization 
at site of the non-replicated system with a given disorder realization J. The equality between the second and the 
third term follows from putting the disorder average outside the sum over the configurations of the replicated system 
and thus recovering the independence of the different replicas prior to the averaging. The above equations tells us 
that the correlation between p replicated spins at the same site is equal to the average with respect of the disorder 
of the p moment of the magnetization at site of the non-replicated system. Note that each disorder realization is 
weighted with a weight proportional to the partition function to the power n. In particular when n we recover 
the standard white average over the disorder while for non-zero n we are selecting samples with free energy different 
from the typical one [3l[. According to section |X] the RS parametrization of the belief 6((To) is obtained through a 
function p{u) (the same for each site) in the form: 

f pl^-^I2a<^0 

'^^^^ - J '^''^(2^;^^^''' ^ ^''^ 

now using eq. ([69)) and ([70| we finally arrive at the following equation: 

p{u) (tanh (3ur du = (72) 

The above equation encodes the physical meaning of the belief function p(u). On a given sample J the magnetization 
mo, J of site is determined by an effective field u = arctanh(mo,j)//3 on site generated by the rest of the system. 
Since the above equation is valid for any p it follows that the junction p(u) is the distribution over the different samples 
of the effective field acting on a given site. 

The same interpretation can be obtained for the belief of the couple of points and for the plaquette. On a given 
sample the effect of the rest of the system on a couple of spins cti and (T2 generates an effective local Hamiltonian of 
the form —{Uaia2 + uiui -|- U2(72) that determines completely the magnetizations and correlation of the two spin; the 
belief function of the couple of points P{U, ui, U2) is the distribution over the different samples of the effective local 
Hamiltonian. 
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The CVM approach (in particular eq. ([5])) tells us that the distributions p{u) and P{U, ui, U2) can be expressed in 
terms of the message functions q{u) and Q{U, wi, U2) in the following way (we specialize as before to the 2DEA model 
on the regular lattice): 

p{u) = / ]^[g(ui)(iMi](5(u - (mi + U2 + "3 + "4)) (73) 



and 



3 3 



P{U,Ui,U2) = J Y]_[li'^i}dUi]Y\_[Q{Vi)dVt] Y\_ [Q{Ua,Ua,Va)dUadUadVa]P{J)dJ X 
2—1 i=l a— up, down 

X d{U ~ {Uup + Udown + J)) 5 ^Ml - (^ Ui + Uup + Udown)^ 6 {^2 - (^ Vi + Vup + Vdown)^ (74) 

In the Bethe approximation the message function q{u) can be interpreted as the disorder distribution of the effective 
field w on a given spin a with connectivity c when c — 1 links connected to it are removed. This is a peculiar feature of 
the Bethe approximation but in general the message functions do not admit an interpretation as distributions as the 
one derived above. Indeed, as we have seen in the previous section, they are not positive definite in general. On the 
other hand it can be argued that the distribution Pno-j{U, ui, U2) of the effective Hamiltonian of a couple of points 
in the absence of the link connecting them obeys the equation: 

P(f/, ui, U2) = j dU'dJPno-jiU',ui,u2)SiU - [U' + J))P( J) (75) 

and therefore Pno-j{U, ui, U2) is equal to ([7^ without the integration over P{J)dJ: 

» 3 3 

Pno-j{U,Ui,U2) = / ]^[g(Ui)'^^'j] Il[9(^^i)c'^'»] Y]_ [Q{Ua,Ua,Va)dUadUadVa] X 
i—1 i—1 a— up, down 

X S{U - {Uup + Udown)) S iui- (^ Ui + Uup + Udown) 5 i U2 - (^ Vi + Vup + Vdown) (76) 



A similar expression can be obtained for the distribution of the effective Hamiltonian of the plaquette without the 
links inside it. We note that the distribution Pno-j{U, ui, U2) allows a straightforward computation of the local energy 
through the following expression that can be also (consistently) obtained in way similar to eq. (|72[) deriving explicitly 
the replicated free energy. 

T? futn^iu iTj \ATT^ ^ tanh(/3t/ + /?J) + tanh/3uitanh/3M2 

E=-j P{J)dJP,,o^,,{U, u,,U2)dUdu,du2J ^ ^ ^^^^^^^ ^ ^^^^^^^ ^^^^^^^ (77) 

From eqs. (j73p and (j74p we see that the message functions Q{U,ui^U2) and q{u) need not to be positive definite but 
they must be such that p(u),P([/, ui, W2) and P„o_j(J7, ui, U2) are. We will check that this is indeed the case in the 
following. In the last section we have argued that at the critical temperature of the spin-glass phase transition the 
fields u are small and at first order in T — the message functions are described by the variables (a, aii{U), ai2{U)) 
that represent the average of the second moments of the fields, see eq. ([56)) and (l58l) . Consistently we noted that 
these variables are proportional to the eigenvector with zero eigenvalue of the Jacobian matrix and checked that the 
eigenvector (and thus {a,aii{U),ai2{U))) is such that the function Q{U,ui,U2) cannot be positive definite, see fig. 
1121 In the following we consider the similar quantities for P{U, ui, U2) near the critical temperature: 

qn{U)= P{U,ui,u2)ulduidu2 (78) 



qi2{U) = J P{U,ui,U2)uiU2duidu2 (79) 

Given that P{U, ui, M2) is a distribution we must find qii{U) > and qii{U) — qi2{U) > for all U. On the other hand 
from eq. (|74p we see that qii{U) and qi2{U) can be obtained from (a, aii{U), ai2{U)) through a linear transformation. 
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again modulo an unknown positive constant that scales as T^ — T. In fig. [T3land[T4lwe plot qii{U) and qiiiU) — qi2{U) 
for the 4DEA model with bimodal interactions slightly below the critical temperature, they were obtained through 
a linear transformation from the corresponding (a, aii(J7), ai2([/)), (aii([/) is shown in fig. W2\ . Consistently we see 
that unlike aii(f7), both qii{U) and qii{U) — qi2{U) are positive for all U as they should. One can check that similar 
quantities corresponding to Pno-j{U, ui, U2) or the plaquette are also compatible with the general fact that the beliefs 
have to be interpreted as a distribution functions over the disorder of the local effective Hamiltonians. 

qii 




FIG. 13: The function qii{U) — J P {U , u-i , U2)u\ dui du2 (modulo an unknown positive constant scaling as Tc — T) at a generic 
temperature slightly below Tc — 2.2 for the 4DEA with bimodal interactions, see text. It is always positive as it should since 
P{U,ui,U2) is a distribution. 




FIG. 14: The function gii ((7) — gi2(f7) = ^ P{U,U\,U2){ui — U2Y /2du\du2 (modulo an unknown positive constant scaling as 
Tc — r) at a generic temperature slightly below Tc — 2.2 for the 4DEA with bimodal interactions, see text. It is always positive 
as it should since P(U,ui,U2) is a distribution. 



In the previous section we noticed that interpreting the message function Q[U,ui,U2) as a distribution of the 
messages (J7, ■ui,M2) on a given instance is wrong and misleading. We have seen why it is misleading: had we 
interpreted in that way we would have look for a solution in the spin-glass phase with strictly positive aii(f/) while 
the true aii{U) is negative form some values of U . We conclude this section providing some arguments to show that 
it is inconsistent. 

Consider the function for the belief of the single point eq. ([75)) . Can we interpreter it as saying that q{u) describes 
the distribution of the messages Ui on a given realization of the disorder? The answer is no because it would lead 
to conclude that these messages are spatially uncorrelated. This is true in the Bethe approximation but not in the 
plaquette approximation, for instance the messages coming from the direction North and West come from regions that 
are both contained in the NW plaquette and cannot be uncorrelated. Furthermore we may notice that this equation 
remains the same if we consider the CVM approximation in which the basic region is a generic L x L plaquette. 
Once again on a given instance we would have four messages entering on a point but when we average over different 
samples these messages are correlated: thus it is crucial to understand that the function q(u) has nothing to do with 
the distribution of the messages on a single sample. Actually the messages are auxiliary objects of the approach while 
the true physical objects are the beliefs. 
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Similarly looking at the equation for the beliefs of the couple of points eq. (l74l) we see that if we interpreted 
Q{U,Ui,U2) as the distribution over different samples of the messages fields {U,Ui,U2) and q{u) as that of u, we 
should have concluded that the corresponding messages are spatially uncorrelated and again we see that this is in 
contrast with the key CVM assumption that objects in the same basic region (the plaquette) are correlated. 

Finally if we go back to the definition of the messages functions we see that a generic message p{(Ji,a2) in terms 
of replicated spins must be positive function, instead the corresponding function Q{U, ui, U2) is some kind of integral 
transform of p(ai, (72) and need not to be positive. 

We are now in position to discuss the relationship between the replica CVM approach and the earlier results of the 
Tohoku group |12Hl4j. In 1980 Katsura, Fujiki and Nagahara were the first to apply CVM ideas to spin-glasses and 
studied the phase diagrams of various models. They started from the CVM message-passing equations on a given 
sample and introduced the functions q{u) and Q(U,Ui,U2) intended to be the sample-to-sample distributions of the 
messages (actually they wrote Q{U,ui,U2) — Q{U)g{ui)g{u2))- As we discussed in section HVl these assumptions 
lead to the same set of equations for q{u) and Q{U,ui,U2) that we have obtained through Replica CVM at the RS 
level. However, even if the equations are the same, the starting assumptions are inconsistent and in the end the 
actual solutions q{u) and Q{U,ui,U2) turn out not to be distributions. Thus we think that in general Replica CVM 
provides a more satisfactory and consistent derivation of the RS CVM equations obtained nearly thirty years ago 
by the Tohoku group. Most importantly the equations follow from a variational principle and can be generalized to 
include RSB. 

Actually Katsura et al. did not write down the full set of equations P3l34l35p but studied the paramagnetic solution 
in various models and considered the location of a second-order spin-glass phase transition. They solved equation ([55)1 
approximating Q{U) as a couple of delta functions, then in order to locate the transition temperature they studied 
the Jacobian under the assumption that Q(U,ui,U2) ~ Q{U)g{ui)g{u2). This assumption is inconsistent with the 
equation for Q{U,ui,U2) that does not admit a factorized solution but simplifies the computation of the Jacobian 
because one has to deal with just two variables a = J q{u) du and an = / g(u) v?' du compared to the full set of 
variables (a, aii(C/), ai2([/)) of the exact plaquette CVM. As expected their results are less precise than ours, for 
instance they obtained a spurious spin-glass phase transition on the 2DEA on the square lattice while remarkably we 
find no second-order phase transition. On the other hand their treatment allows to obtain an analytical expression of 
the Jacobian and of the phase diagrams and they could apply it to various models. 



VIII. DISCUSSION 



The Replica CVM can be used in principle to study the phase diagram of a wide range of disordered systems and it is 
also possible that it can be used to get quantitative predictions on single samples. However there are quantitative and 
qualitative technical differences with respect to standard CVM (and the GBP algorithm), to the Bethe approximation 
and to SP that represent a great challenge. In particular, integrals in many dimension, convolutions and message 
functions that are not positive definite conjure to make applications extremely difficult beyond the simplest CVM 
approximations and smallest number of RSB. 

Besides these technical difficulties the results of the second part of the paper show that, at least in the averaged 
case, the method yields sound results. This is not at all trivial given the frustrated nature of the model and the tricky 
continuation in replica number. 

In many respects the method shares the same advantages and weak points of standard CVM. It is a good tool to 
obtain quantitative non-perturbative estimates in actual models and to characterize phase diagrams, e.g. the stability 
of the paramagnetic phase down to zero temperature in 2D spin-glass models is a remarkable result and we already 
mentioned that the Jacobian approach can be straightforwardly applied to study the ferromagnetic transition in 2D 
EA models with ferromagnetically biased interactions. Thus the method improves over perturbative schemes, say the 
1/^ expansion that on the other hand are more tractable in order to get qualitative results on systems with full-RSB 

On the other hand a standard CVM approximation is not guaranteed to be an upper bound to the true free energy 
and like many non-perturbative approximations can possibly give inconsistent results, in this respect we note that 
the plots of the free energy suggest that a good sequence of Replica CVM approximations of increasing precision 
should approach the true free energy from below and not from above as in standard CVM. This appears to be a 
consequence of the replica trick much as the fact that Parisi formula for the SK model has to be maximized and not 
minimized in order to determine the free energy ,15{j. Another drawback is that CVM is intrinsically a mean-field 
approximation. As such it will always predict mean-field critical exponents although in principle information on the 
true critical exponent can be obtained [isj e.g. comparing approximations with maximal regions of increasing size. 
However the problem of going beyond mean-field theory {i.e. using the loop expansion in Replica Field Theory) is way 
more important than to build more precise mean-field approximations like those yielded by replica CVM. This said 
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quantitative estimates can be rather useful to help solve long-standing problems in spin-glass theory. For instance an 
estimate of the actual location of the De Almeida- Thouless line [l^ in finite dimensional models is highly desirable. 

The next step beyond the applications presented here should be the RS treatment of the spin-glass phase of the 
EA model in the plaquette approximation, at least in D > 3. This include the extension to finite small n that 
can be seen as a first simplified version of IRSB (the so-called factorized solution). On the Bethe lattice [i^ this 
improves considerably the RS result and differs little from the more precise IRSB solution [2]. A similar considerable 
improvement of the factorized solution with respect to the RS one also in the plaquette approximation, if observed, 
would constitute non-trivial evidence supporting the RSB nature of the spin-glass phase in finite-dimensional models 
below the upper critical dimension 6. 

A way to reduce the technical complexity of the exact Replica CVM equations is to use the variational expres- 
sion, eqs. ([7]) and (jB13[) . of the free energy parametrizing the messages functions in a simplified way and solving 
the corresponding variational equations that will be different from the exact equations (jB2IB3l) and (IB4IB5p . The 
number of possible parametrizations is virtually infinite, for instance it should be noted that in high-dimension the 
plaquette approximation can be considered a perturbative correction to the Bethe approximation and the RS function 
Q{U,ui,U2) is peaked at small values of all its arguments. Therefore in that regime it is consistent to parametrize 
Q{U, Ml, U2) with few of its moments while considering the full q{u), the application of this ansatz to finite dimension 
would be non-perturbative but could give consistent results. A different possibility is to assume that in the whole 
low-temperature phase, not only near Tc, the small fields distributions are essentially parametrized by their second 
moments, this amounts to consider as variables the functions Q{U) and the parameters {a,aii(U),ai2{U)) introduced 
in section IVII To obtain the variational equations of this ansatz we should parametrize the distribution of the fields 
{U,Ui,U2) as delta functions plus second derivatives of delta functions, such that all moments of order higher than 
two vanish. 

The main reason to look for tractable replica CVM approximations beyond the Bethe approximation is that one 
could then try to apply them on single samples in the spirit of the SP algorithm. On the other hand if one is just 
interested in the averaged properties it may be useful to consider also tree-like approximation with regions of increasing 
size. Being tree-like they are free of the convolutions and negative message functions problems, i.e. fully treatable 
with population dynamic algorithms. Therefore although they will be less precise than CVM approximations larger 
regions could be treated and extrapolation to the actual finite dimensional model achieved. 
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Appendix A: The Hierarchical Ansatz 

In this appendix we present the hierarchical ansatz for the replicated CVM in full generality. In particular we will 
consider: i) a general CVM approximation, ii) a general number of RSB steps, iii) both the single sample and averaged 
cases. In appendix |B] the variational equations and the variational free energy will be derived. 

In section IIIII we have written the free energy in terms of messages that are positive functions of the configu- 
rations Xr of the variables in region r. In the following we will parametrize the messages in order to be able to 
take the analytical continuation to real n, the RSB parametrization requires the introduction of a set of K RSB 
parameters 1 < xi, . . . , xk < n that are numbers. Unfortunately at this stage the standard notations of the message- 
passing formulation of CVM Q and of RSB [l^ overlaps and the reader should not confuse the RSB parameter 
1 < xi, . . . ,xk < n (that are numbers) with the Xr defined previously (that specifies distinct configurations of the 
variable nodes in region r). However from now on we will concentrate on the Edwards- Anderson model defined above, 
therefore the nodes are Ising spins, e.g. for the region comprising spin a and b we have Xab = {ca, crt} ff the system 
is not replicated and Xab = {o'a, S't} if the system is replicated n times, such that at each site, say a, we have n spins 

da = (al,... a^). . , . . . . [-^ . 

The general RS and RSB ansatz of a function p{a) of n spins was originally presented in [19| , later its parametrization 
in terms of distributions of fields was suggested in [20j and later revisited in and we refer to those paper for an 
explanation of the main ideas underlying it. Here we generalize it to a generic function p((7i, . . . , ap) where each ai is 
a set of n Ising spins. 

We start introducing the field U that parametrize a probability distribution of p Ising spins, in the following we 
call it a p-field. [/ is a set of 2*^ — 1 real numbers {uj} where / is an index that labels all the subsets of the set of 
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indices {1, . . . ,p}, (the empty set is excluded). We have then: 



Pu{(Ji, ■.■,<Tp) ^M{U)exp/3 



^ UifTi + ^ MijCr,;(Tj ■ + ^ UijkaiajCTk + 1- wi,...^pCri . . . (Tp 



(Al) 



Where M{U) is a normahzation constant. We define a probability distribution (population) P^'^^U) of such fields a 
0-distribution, correspondingly a 1-distribution is a probability distribution on probability distributions (population 
of populations) and so on. A fc-distribution will be written as P'^^^ and it defines a measure over the 

space of fc — 1-distributions. 

In order to parametrize the function /o(a'i, . . . , dp) with K steps of RSB we need: 

• a iiT-distribution p(^) of the fields U; 

• K integers 1 < xi, . . . , xk < n (for n < 1 they become real and the inequalities change sign); 

In the following we will consider the parameters 1 < xi, . . . , xk ^ n fixed and consider just the dependency on the 
distributions. The construction is iterative and requires a set of functions pp(k) (cti, . . . , dp) where each di is a set of 
Xk+i spins with k ^ 1, . . . , K (we define xk+i = n and xq = 1). The normalization of pp{k) is crucial, we choose to 
normalize all of them to 1. We define pp{k) (cti, . . . , dp) starting from pp{k~-i) {di, . . . , dp) first dividing all the Xk+i x p 
spins in Xk+i/xk groups {di, . . . ,dp}c of I'fc x p spins labeled by an index C = 1, . . . , Xk+i/xk- Then we have: 

/Xk+l/Xk 
pWrfp(fc-i) pp,k-,ji{di,...,dp}c) (A2) 

c=i 

Thus p{di^ . . . , dp) = pp(K) ((Ti, . . . , dp) is defined iteratively starting from the Replica- Symmetric case corresponding 
to /e = 0: 

/xi 
P^°\U)duY[PuiWi,...,ap},) (A3) 

We will parametrize each message m^s {xg ) through a population P^i^^ over a p- field where p is the number of sites 
in region s. The populations associated to the tilded messages rhrs defined in eq. © will be represented by Prf'\ 
Note that at integer values of n, the above parametrization is redundant, and in principle we cannot determine the 
population knowing the messages. It is standard in the replica method to assume instead that this step can be 
performed due to the continuation to real values of n. 

In the end we will write down the message-passing equations in terms of the message populations and thus we 
will be able to consider non-integer values of n, and in particular the limit n — > 0. We will also obtain a variational 
expression of the free energy in terms of the message populations. We note that in principle one could parametrize the 
beliefs with populations and obtain an expression of Kikuchi free energy in terms of these populations. The resulting 
expression however is extremely complicated due to the presence of the entropic terms in the form br In br, see e.g. 
[20| where the entropy of the point belief was computed. The main reason why we derived the variational expression 
d?]) in terms of the messages is precisely because it avoids to deal with terms of the form ^^r^^K- 

The r.h.s. of the rescaled iteration equation pO|) defines a function rhrs of the messages in M{r) \ M{s). Now since 
the messages are parametrized by iiT-populations, we need to determine the corresponding function Pi^^ that yields 
a -fC-population as a function of the A'-populations of the messages, such that the iteration equation translates into 

rrs — ^ rs . 

We start introducing two J-dependent functions riij^rs and Nj^^s such that the following equation is satisfied: 

'^J,rs^J,rs = ^lki\s{Xr) W tTlr" s" (A4) 

Xr\s m^ii ^11 eM(r)\M(s) 

We use bold face for indicating functions of messages, but we stress that while vh.j^rs returns a normalized probability 
on spins in region s, the function Nj.rs returns a number which is precisely the normalization required by equation 
(|A4[) . This equation can be written in terms of populations and defines a if-population Pj^^ as a function of the 
iiT-populations in M{r) \ M{s). It turns out that the function Pj^l, can be defined in terms of the function Pj^^ ^' 



26 



that in turn can be defined through the function Pj^^ and so on. The resulting expression for a given k is: 



^J,rs \m^„^„GM(r)\M(s) / 

where Nj'^^g is a number that is a function of the /c-populations P,j:!9^,, in Af (r) \ M{s) defined according to: 

^ / I n ^J-I-rf^J-^'M [n£-/¥-^/^^ . (A6) 

\m^„,„eM(r)\A/(s) / 

In the cavity formulation at the Bethe level [1] the quantities N^^J^ are associated to (cluster) free energy shifts and 
they appear in (jA5l) as reweighting terms. At the Bethe level, explicit expressions like (jA5[) for any k value have been 
already reported in Ref. [l^l • 

The iterative definition must be supplemented with the two functions for fc = and fc = — 1. They reads: 

fi n P!^UUr"s" ] [^^j:rh^^S{Urs - U,,..) (A7) 

^J,rs •' \m^,,^„eM(r)\M(s) J 

and 

(0) = / I TT p(n) .. .. \ r^(-i)i-i 



^ts - / n PlZ-dU."s- [Nfctt^ (A8) 

\m^j,^„£M{r)\M{s) J 

For xi = n this yields the Replica- Symmetric solution, note that in this case in the n — )■ limit the reweighting term 
goes to 1 and is irrelevant. In the above equations Urs and N^s are functions of the fields Ur"s" that have to be 
obtained solving the following single-replica equation: 

Pu,,„(^.)N|,:/i= ^ <,(fT.) n Pu^».,,M (A9) 

{ct^\s} m^„^„eM(r)\M(s) 

Note that it is only at this stage that the actual properties of the model enter. This complete the recursive definition 
of the function Pj^l^ and Nj^^^that solve equation (jA4p at any level of fc-RSB, we will not write down the proof of 
this statement that can be worked out iteratively generalizing a similar derivation in the Bethe approximation plj . 
It is important to stress that this iterative definition is possible only if we do not average over the disorder, a passage 
that will be taken in the next section. 

Up to now we have expressed eq. (|10p in terms of populations in such a way that the n — > limit can be taken, in 
the following we consider the similar treatment for eq. Note the basic differences between eq. © and (|10l) : the 
absence of the summation over the spins in region r\s and the fact that we consider messages in M(r, s) and not in 
M{r)\M{s). 

Much as above, the function q[s'' corresponding to the r.h.s. of eq. (|9]) can be defined in an recursive way, the 
result being: 

Ql? = -4y / H^^dPjt'' I n P!Z'dP^':A [Mi1-^n^^^^/^^5{Pjt'' ~ Qil-^y) (AlO) 



~ (k) 

where Mrs is defined as: 



M« = / PlfdPl";-^^ I n Pr'l'dP,%P I [Mlfl-'^f"^'/'"' . (All) 

As before the previous iterative definition is completed specifying the two function at the Replica- Symmetric level 
corresponding to fc = 0: 

Q?J = -4) / Pl-'^^dUrs ( n P^'ldU,.'s" I [Mi-'^r6{Urs - Qi-'^) (A12) 
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and 

^ / P^'^dUrs I n P^r^l'dUr^'A [MI^D]- . (A13) 

For xi ^ n this gives back the Rephca-Symmetric solution, note that in this case in the n — ^ hniit the reweighting 
term goes to 1. 

The two quantities Qf^^'' and Mr^ have the foUowing form that does not depend on the Hamihonian of the 
problem: 

The above sum is intended in vectorial form, i.e. for each possible combination of the spins in region s 
{ai,aiaj,aiajak . ■ ■) we sum the corresponding fields. The normalization factor can be written as the product 
of terms depending on each message separately times a term depending on the sum of the fields: 

Ar(Qfc") 

The above properties have important consequences and allow to introduce Fourier-like transforms in order to write 
down the equations for the messages in explicit form, see appendix [Cl 



Appendix B: Generalized Survey Propagation Equations 

We are now in position to write the variational equations ([B]) in terms of iiT-populations. We start noticing that in 
the averaged case eq. (|10l) can be written as: 

m^sC = (mj,rsN,7,rs)j (Bl) 

where C is a normalization constant. The r.h.s. of the previous equation is a linear combination of spin probabilities 
and it is easy to see that the corresponding if-population is just a linear combination of the corresponding K- 
populations with same coefficients Nj^^s and the normalization factor is just that sum of the coefficients i.e. (Nj^^s) j- 
The populations corresponding to m,/,rs are given by eq. (jA5[) and we see that the coefficients Nj^^s in eq. (jBl|) 
cancel the term at the denominator in (|A5p . thus the resulting equation for the populations Pr^^ is: 

P(f ) ^ — iy- / ( n H^UHff^A ([Nfj^)]--/--5(p(f-) - Pf-'^))j (B2) 

V^J.rs/J \m^„^„eM{r)\M{s) J 



P^rf' = Q^f ^ (B3) 

Note that in the limit n — >■ we have Nr^"* — 1 because xk+i = n. These equations are translationally invariant, 
meaning that the populations Pr^'^ does not depends on where regions r and s are actually on the lattice but just 
on their shape and mutual positions with respect to each other. For instance in the Bethe approximation we have a 
single if-population that does not fluctuate over the sites. The right hand sides of the above equations (IB2p and (jB3[) 
should be thought of as respectively the r.h.s. and l.h.s. of a single equation that in general involves the messages 
populations Pr^"^ in implicit form. This is the main difference with respect to the Bethe approximation where the 
messages appears in explicit form and the equations can be solved iteratively, a discussion of these equations in the 
context of the CVM plaquette approximation will be given at the end of next section. 

On a single sample instead the distributions fluctuate over the sites. As a consequence the solution with if-RSB 
steps is parametrized by (K — 1)-RSB populations fluctuating over space. The corresponding equations are: 
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P^r'' = Q^f (B5) 

For instance the 1-RSB solution on a given sample is described by populations of fields that fluctuate over different 
regions. Again the right hand sides of the above equations (jB4[) and (jBSp should be thought of as respectively the 
r.h.s. and l.h.s. of a single equation that in general involves the messages in implicit form. 

When K = 1 and the maximal region is the couple of points (i.e. we work in the Bethe approximation) the above 
equations reproduce the celebrated Survey Propagation (SP) equations [11-0] j hence the name Generalized Survey 
Propagation equations. The SP equations were originally obtained through the cavity method, while a derivation of 



the SP equations on a single sample using replicas was obtained in [51 1 



1. The Free Energy and its Derivatives 

The variational expression of the free energy eq. ([7]) can be written in terms of populations along the same lines of 
the previous sections. The results in the averaged case is: 

FK = -J2^r\n{-N%^)j (B6) 

reR 

where Nj^'' is a J-dependent function of the ii'-Populations corresponding to the messages in M(r) that can be 
defined iteratively in terms of the populations as: 



S-/ n p.^'dP^'^'H-^V^r-^'^'''' (B7) 



with 



r"s"eM(r) 



''"'-J n p!^.'dUr"s"(^^'r 

r" i-" (Z AT ( 'r^\ 



^J.r= P.^,i„dUr--s"{NZ"r (B8) 



r" s'' eM{r) 

where 

Nj;'^ = E ^-^('^-) n Pu.",'(''s") (B9) 

{cTr} r"s"eM(r) 

On a single instance instead we have: 

FK = -Y.Cr\nNf-''> (BIO) 

reR 

where the Nj^ have to be evaluated in terms of the K — 1-populations entering region r. In order to make explicit 
the similarity with the cavity formulation of [l| we define a new function 

Consequently the above quantity has again a recursive definition: 



= In / n P^'UP^'l'^e-^^^^^^^'^ (B12) 

r"s"tEA/(r) 



with this definition we have: 



and 



Hn) = -^Y.^rHe-'-''^'^')j (B13) 



^-lTo*(")-iE^J n Pi^]M^J,'\^Ff-'')j (B14) 

reR r"s"eM{r) 
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In the average case the message populations do not fluctuate over space and the contribution of each region of a given 
form to the free energy is the same. Thus the sum over all regions can be replaced by the sum of the contributions 
of the basic types of regions r each multiplied by the number rir of regions of type r per spin. For instance in the 
plaquette approximation of the 2D square lattice we have Cpiaquctto = 1, Ccoupic = ^ 1 and Cpoint = 1, and the number 
of regions per spin are respectively npiaquette = 1, "couple = 2 and npoi„t = 1. 

It is also important to determine the derivatives of the free energy with respect to the parameters 1 < , . . . , xk < n 
[l|, [2| ■ Since the above expression is variational with respect to the populations the total derivative of the free energy 
with respect to coincides with its partial derivative with respect to Xk- 

Clearly in order to determine the derivative of the free energy with respect to the parameters 1 < xi , . . . , xk < n 
we only need to determine the derivatives of the function AFj^-*. The derivative of ^F^fl with respect to Xp^i is: 

9.^,, AFg = --^Af£ + {{AF^J;'%i (B15) 



where we have defined: 



J llr"s"eM(r)^r"s""-^r"s" ^ 

Then the derivative d^^^^^A^^^^ can be obtained using the recursive definition. Indeed we see that the derivatives of 
AFj^ for k > p with respect to Xp+i parameters obey the following recursive equation: 

a.^^, AfS = AF,t"^'))'^,^ for k > p (B17) 

For instance in the IRSB case in the n -> limit we get a similar result to Ref. [l|: 

^ rS-R r"s"GM{r) ^ ^ ^ 



2. From populations to functionals 



In the preceding sections we have worked under the assumption that the messages and the beliefs are both 
parametrized through populations of populations of fields. In section IIVI we have applied the approach to the EA 
model on various lattices in the CVM plaquette approximation. We made the simplest non-trivial ansatz, i.e. the 
RS ansatz, that we expect to be good in the high-temperature paramagnetic phase. On a single sample the messages 
are just numbers in the RS phase, and the corresponding equations are the GBP equations of [1]. For the replicated 
averaged system instead the messages according to the above parametrization should be distributions of fields. As we 
have seen it turns out that the actual solution requires to consider messages parametrized by functions that are not 
positive definite i.e. that are not distributions. This tells us that in practice when solving a given model we may be 
forced to consider parametrization of the form where instead of populations of fields we should consider functions 
of the fields, or instead of populations of populations we should consider functionals. At any rate, it is easily seen that 
the equations obeyed by these objects do not change at all depending on whether they are distributions or functions 
i.e. all the results of the previous subsection hold. On the other hand it can be argued that the function Nj^^ defined 
above should always give a positive number and this is guaranteed only if it is a function of /c-populations, therefore it 
appears that in any case we should parametrize a message at level K of RSB with a function oiK ~ 1-populations and 
this function may eventually be non-positive definite. In other words it seems to us that we can relax the condition 
to use distributions only at the last level of RSB. 



Appendix C: The Inversion Problem 



The generalized survey propagation equations derived previously express the tilded populations in terms of the 
non-tilded ones. To obtain an explicit expression for the message functions one needs to invert equations (|B3I) (or 
eq. (jB5|) on a given sample) and obtain an expression for the non-tilded populations in terms of the tilded ones. At 
the lowest RSB levels, (i.e. RS in the averaged case and IRSB on a single sample) this can be done using Fourier 
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transforms. In this appendix we will show that at any number K of RSB steps the inversion can be achieved in 
principle using appropriate integral transforms of the populations. An algorithm able to go back and forth from the 
populations to their transforms would provide a route to the numerical solution of the Generalized Survey Propagation 
equations but unfortunately an efficient implementation seems quite difficult. 

Basically we will work with nested Fourier transform, i.e. Fourier transforms of Fourier transforms. We will 
introduce the invertible integral transform Trs ^ of a /c-population Prs^ and show that eq. (jB3p in terms of transform 
reads: 

r« = r«+ E ^r'l (ci) 

From the previous expression we can easily compute Trs^ from "Trs^ . Again we will proceed iteratively, showing that 
if the previous equation is valid at level fc — 1 it is also valid at level k. In order to do this we also assume that it 
exists a function CjC^'^^) such that the quantity Aii'l can be written as 

j^{k-l) ^ G'^'^^^HTr^s ^^)Um^„,„eMir.s) g^^'^^T^^s"^^ ) ,^2) 

(?(fe-i)(r;r'') 

where "Trs ) in the above equation is computed as the sum of the non-tilded transform. We can write eq. (IA10[) in 
terms of the transforms at level A: — 1: 



r" s" V ' r" s 

M(r,s) 



\ m^,, ^„ eM{r,s) j 

now using eq. (jC2p we can redistribute the reweighting term between the various populations and rewrite the previous 
equation as: 



X 



s\frr''-xr''- E r},^s''] (C4) 

m^v/^// £ A'/(r,s) 



In the previous equation Q^?(g^s defines a measure over the space of fc — 1 transform. We Fourier transform 

the previous equation with respect to the fc — 1 transform and obtain: 

•^r? = ^(fe) -^r? Y\. (^5) 



~ (k) 

accordingly the factor A4rs can be obtained as the transform evaluated at a given value, e.g. zero argument; this 
yields: 

Now we can take the logarithm of the previous equation and define: 

r. 



%(^^)^\n^^ (C7) 



we see that with this definition eq. (jCip is satisfied at level fc. Correspondingly eq. (|C2|) is also satisfied provided we 
identify the function Q with: 

g(?^J-W(0) (C8) 
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To complete the proof we need to show that eq. (jCl|) and eq. (jC2l) are vahd at = 0, the replica symmetric case. To 
do this we easily see that eqs. (jA14p and (jA15[) have this structure. Therefore at the RS level the transform TrT' is 
defined as: 



/ P^°\UrsW{UrsT'dUrs expiSrsUr 
JP!^s\UrsWiUrs)'='dUrs 



{Srs) - In ^^^^^^ (C9) 



5l°^ = / P^^HUrsWiUrsy'^'dUrs (CIO) 



and we have: 



Summarizing, at the RS level, the integral transform needed to solve the inversion problem is defined as the logarithm 
of the Fourier transform of the reweighted distribution of fields. Note that if the solution is purely RS we have xi = n 
and the reweighting term is absent in the n — > limit. From the above RS expression one can go to IRSB using eq. 
(|C7p . At IRSB the solution is parametrized by a population of populations, and the transform is the logarithm of 
the Fourier transform of the reweighted distribution of the logarithm of the Fourier transforms of the populations. 
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